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Abstract 

We study generating functions for the number of involutions, even involutions, and 
odd involutions in S n subject to two restrictions. One restriction is that the involution 
avoid 3412 or contain 3412 exactly once. The other restriction is that the involution 
avoid another pattern r or contain r exactly once. In many cases we express these 
generating functions in terms of Chebyshev polynomials of the second kind. 

Keywords: Restricted permutation; restricted involution; pattern-avoiding permu- 
tation; forbidden subsequence 



1 Introduction and Notation 



Let S n denote the set of permutations of {1, . . . , n}, written in one-line notation, and suppose 
7i G S n . We write \tt\ to denote the length of n, and for all i, 1 < % < n, we write n(i) to 
denote the zth element of it. We say n is an involution whenever 7r(7r(i)) = i for all i, 
1 < i < n, and we write I n to denote the set of involutions in S n . Now suppose 7r G S n 
and a G S^. We say a subsequence of tt has type a whenever it has all of the same pairwise 
comparisons as a. For example, the subsequence 2869 of the permutation 214538769 has 
type 1324. We say n avoids a whenever n contains no subsequence of type a. For example, 
the permutation 214538769 avoids 312 and 2413, but it has 2586 as a subsequence so it does 
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not avoid 1243. In this context a is sometimes called a pattern or forbidden subsequence 
and 7r is sometimes called a restricted permutation or pattern- avoiding permutation. In this 
paper we will be interested in involutions which avoid several patterns, so for any set R of 
permutations we write I n (R) to denote the set of involutions in S n which avoid every pattern 
in R and we write I(R) to denote the set of all involutions which avoid every pattern in R. 

In P Egge connected generating functions for various subsets of 7(3412) with continued 
fractions and Chebyshev polynomials of the second kind, and gave a recursive formula for 
computing them. For instance, he showed [U Theorem 3.3] that 



e n 



X 
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2 

X-i Xo 

7re/(3412) k>l I — X\ 
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a^j^J^2*^'3'^4*-' j 5 

2n /2n-2\ , /2n-l\ 2n+l I 2n \ 

Here the nth numerator is n x i * _1 an d the nth denominator is 1 — Y[ x i ■ 

i=i i=i 

Moreover, we have used the notation we will adopt throughout the paper: we write [k] to 

denote the permutation k ... 21 and we write r k (ir) to denote the number of subsequences of 

type [k] in ir. Egge also showed [U Theorem 6.1] that for all k > 1, 



<jeJ(3412,[2fc]) 



and 



x \a\ = U k~l (jf) + ) 
a e /(3412,[2fe-l]) X ) + ^fc" 1 (^)) 

Here U n (x) is the nth Chebyshev polynomial of the second kind, which may be defined by 
sin((n + l)t) 

U n (cost) = . Egge proved these results using the following recursive formula jU 

suit 

Corollary 5.6] for the generating function F^(x) = x'' 7 ', which makes it possible to 

o-6/(3412,7r) 

compute F+(x) for any permutation tt: 

k 

F+(x) = 1 + xF+(x) + x 2 J] (^re^(^) - *±s=^*)) (4) 

t=i 

Here the various subscripts of F + on the right are the types of certain subsequences of tt. For 
other results concerning pattern-avoiding permutations, continued fractions, and Chebyshev 
polynomials, see El El and the references therein. 

In this paper we refine Egge's results by studying generating functions for even and odd 
involutions in 7(3412). For any permutation tt, we write sign(n) to denote the sign of 7T, 
which is 1 if tt is an even permutation and — 1 if tt is an odd permutation. Using Egge's 
techniques, we prove signed analogues of (QJ-fjU), from which one can obtain analogues for 
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even and odd permutations. For instance, we show that 

\sign(w) TT r k (n) _ 



(-l^MJJrr, 



2 

7r6/(3412) fc>l 1-XiH '' L ''- 



2 5 4 

np JU rp'-' rp^ ry 

I _ T1 T 2 Tq + gj^gggj 
l XiX 2 X3 f 1 4 g 4 



1 4 4 

X X^X^X^X^X^ I 

2n /2n-2\ + /2n-l\ 2n+l / 2n \ 

Here the nth numerator is FJ 1-1 1-1 and the nth denominator is 1 — ]~[ x 4 . We 

i=i i=i 
also show that for all k > 1, 



<rel(3412,[2fc]) 



sign(Tr) \cr\ k—1 \ 2j x ) 

' *u k (^) 



and 

^(34^2,-1,) -(^(^)-^lte))' 

These results follow from our analogue of (J3J), which is the following recursive formula for 
the signed generating function F~(x) = ^ (— l) st9n ^x^: 

cre/(3412,7r) 



K 

F~(x) = 1 + xFlix) - x 2 V ( f~ m (x) - F- 



Xj I ^!v.m...mrt, (XJ 



i=l 

This result makes it possible to compute i^(^c) for any permutation ir. 

In addition to signed generating functions for various subsets of 7(3412), we also study 
generating functions for involutions which contain a given pattern exactly once and avoid an- 
other pattern. For any patterns <7i and o<i and any n > 0, we write I n (&i] cr 2 ) (resp. I{<Ji, (J2)) 
to denote the set of involutions of length n (resp. of any length) which avoid o\ and contain 
exactly one subsequence of type <r 2 . We first use Egge's description of the recursive structure 
of 1(3412) to obtain recurrence relations for the generating functions G^(x) = 

o-e/(3412;7r) 

and G~(x) = ^_iy % 9n{°) X W\ f or cer tain tt. Using these recurrence relations we find 

o-e/(3412;7r) 

G+(x) and G~(x) for various tt in terms of Chebyshev polynomials of the second kind. For 
example, we show that for all k > 3, 

G U /I Ol Q 



and 



G 1, o fx) 



x(l — X + X 



2\ 



and we find similar results when 213 is replaced with any permutation of length three. 
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We then turn our attention to the set 7(; 3412) of involutions which contain exactly one 
subsequence of type 3412. We first describe the recursive structure of 7(;3412), which we 
use to find the signed and unsigned generating functions for J(;3412). We then use this 
structure to obtain recurrence relations for the generating functions P^{x) = 

cre/(7r;3412) 

andP~(x)= (— l) swn ^x^. These recurrence relations enable us to find P+(x) and 

cre/(7r;3412) 

P~(x) for various n in terms of Chebyshev polynomials of the second kind. For example, we 
show that for all k > 1 we have 

and 

p M _ E^mm 

U k \ 2ix ) 

where v k (x) = J2j=o ( -1 )® x ° ■ 

We conclude the paper with a short list of directions for future research. 

2 The Recursive Structure of 7(3412) 

In this section we recall the natural recursive structure of 7(3412), which was first observed by 
Guibert Remark 4.28]. We begin with notation for three ways of combining permutations. 
Definition 2.1 Suppose n G S m and a G S n . We write n © a to denote the permutation in 
given by 



[n © a){i) 



ir(i) if 1 < i < m, 

a(i — m) + m ifm + l<i<m + n. 



We refer to tx © a as the direct sum of n and a. 

Definition 2.2 Suppose n G S m and a G S n . We write n a to denote the permutation in 
S m +n given by 

I a(i — n) ijm + l<i<m + n. 
We refer to ir a as the skew sum of n and a. 

Definition 2.3 Suppose ty G S m and a G S n . We write n * a to denote the permutation in 

Sm+n+2 

given by 

7r*a = (10 710 1) ©a. 

We now describe the recursive structure of 7(3412). 
Proposition 2.4 ('11, Proposition 2.8]) 
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(i) For all n > 1, the map 

4_i(3412) — > /„,(3412) 

7T I— > 1 © 7T 

a bijection between 4-i(3412) and the set of involutions in 4(3412) which begin 
with 1. 

(ii) For all n > and a// j sitc/i that 2 < j < n, the map 

4_ 2 (3412) x 4^(3412) — ► 4,(3412) 
(n, a) i — > 7r * a 

a bijection between 4_ 2 (3412) x 4-j(3412) se£ of involutions in 4(3412) 

which begin with j . 

3 Continued Fractions 

In this section we will encounter several continued fractions, for which we will use the fol- 
lowing notation. 

Definition 3.1 For any given expressions a» (i > 0) and b{ (i > 0) we write 

ao Oi «2 «3 

fro + h + b 2 + h + 
to denote the infinite continued fraction 

a 



ai 

Oo + 



a 2 

&i H 



02 H 



a 4 

63 + ITT 

64 H 

For all i > 1, let denote an indeterminate, let x = (xi,x 2 , . . .), and let C + (x) and 
C~(x) denote the generating functions given by 



^fcOO 

7re/(3412) fc>l 

and 



( x ) = j2 ( - 1 ) si9n(7r) n xTfc w • 

7re/(3412) fc>l 

Egge has shown p!J Theorem 3.3] that 



2n (2n-2\ J _(2n-l\^ 



n r v 11 

I i~f^ ry ry*" ,-y»'J /-yi" ,-y> 

1 - xi - 1 - xix^ - 1 - ccizfa^fxs - - 2, 1+ 1 (*\) - 

1 11 X i 

i=l 



In this section we refine this result by expressing the signed generating function C~(x) 
as a continued fraction. Combining this expression with (J3J) leads to expressions for the 
restriction of C + (x) to even (resp. odd) permutations as a sum (resp. difference) of two 
continued fractions. We begin with the following simple observation. 
Lemma 3.2 For all i > 1, let Xi denote an indeterminate. Then 

C"(x) = C + (xi, -x 2 ,x 3 ,x 4 , • • •). (6) 

Proof. Fix 7r G 7(3412). If n is even then it contributes rifc>i %k on both sides of (jBJ) and 
if 7r is odd then it contributes — rifc>i ^fe*^ on both sides of ©• □ 

We now express C~(x) SIS db continued fraction. 
Proposition 3.3 For all i > 1, let X{ denote an indeterminate. Then 

2n /2n-2\ , (2n-l\ 
-1 9 9^4 1 

W ~ 1 - xx + 1 - Xl x 2 2 x 3 + 1 - x^xjxjxjx 5 + " ' + _ 2 n 1 (*-") + 

i=l 

Proof. Replace x 2 with — x 2 in © and use (jOJ). □ 

Using J7J) we can express this signed generating function with respect to various statistics 
ClS db continued fraction. 

Corollary 3.4 For any permutation n, let inv(n) denote the number of inversions in it. 
Then 



signjir) Jnyjir) Jtt\ _ 1 37 g X Q X g 

1 — X\ + 1 — xq 2 + 1 — xq 4 + + 1 — xq 2n+2 + 



7rG/(3412) 



Proof. In (JJJ), set Xi = x,x 2 = q, and x« = 1 for all i > 3. □ 

For our next application of (|7|). recall that a left-to-right maximum in a permutation 7r 
is an entry of n which is greater than all of the entries to its left. Similarly, a right-to- 
left minimum in n is an entry of ir which is less than all of the entries to its right. We 
write lrmax(jT) to denote the number of left-to-right maxima in tc and we write riming) to 
denote the number of right-to-left minima in n. Egge has shown |TJ Proposition 3.7] that if 
7T G 7(3412) then 

oo 

lrmax(ir) = rlmin(n) = ^^(— l) fe_1 r fc (7r). (8) 

k=i 

Combining this with (JJJ gives us the following continued fraction expansion. 
Corollary 3.5 We have 

7rG/(3412) 7re/(3412) 

i 2 2 2 

1 x q x x 

1 — x + 1 — X + 1 — X + + 1 — X + 

6 



and 

o 

\ r / -^\sign(w) lrmax(Tr) ^,\tt\ _ \ / ^\sign(ir) rlmin(ir) ^,\ir\ 

Proof. To obtain the first line, set X\ = xq and Xi = q^ 1 ^ 1 for all % > 2 in (jZj) and use (jHJ) 
to simplify the result. The second line follows routinely from the first line. □ 

For our final application of (J7J), recall that % is a fixed point for a permutation n whenever 
7r(z) = i. We write fix(n) to denote the number of fixed points in n. Egge has shown jU 
Proposition 3.9] that if ix 6 7(3412) then 

oo 

/zx( 7 r) = ^(-2) fc -V fc ( 7 r). (9) 
fc=i 

Combining this with (jZJ gives us the following continued fraction expansion. 
Corollary 3.6 VKe /iat>e 



El r 2 
/ -tyignM n fixM T M — _ 
[ } q X ~ l-XQ + 1- 



XO + l — Xfl+l — XO+ +1 — X(7 + 

7rG/(3412) ill i 

and 

'X 



ST (l\sign(n) n fix(n)\n\ _ ~ 1 + ^ ~ V 1 ~ 2x Q + + 4 ^ 



7rg/(3412) 



Proof. To obtain the first line, set iei = xq and Xj = g( 2 ^ 1 for alH > 2 in (J7J) and use Q 
to simplify the result. The second line follows routinely from the first line. □ 



4 Involutions Which Avoid 3412 and Another Pattern 

We now turn our attention to signed generating functions for involutions in 7(3412) which 
avoid a set of additional patterns. We begin by recalling a method of decomposing permu- 
tations and a map on permutations. 

Definition 4.1 Fix n > 1. We call a permutation it e S n direct sum indecomposable 
whenever there do not exist nonempty permutations tc\ and 112 such that n = i\\ © 7T2 . 

Observe that for every permutation 7r there exists a unique sequence of direct sum inde- 
composable permutations a±, . . . , such that 7r = a± © a<i © • • • © cx-k- 
Definition 4.2 For any permutation tx , we define W as follows. 

1. = andl = 0. 

2- If\n\>2 and there exists a permutation a such that it = 1 a 1 then W = a. 

3- If \n\ > 2, there exists a permutation a such that tt = 1 © a, and a does not end with 
1 then W = a. 
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4- U I 71 "! > 2, there exists a permutation a such that n = a © 1, and n does not begin with 
| -7T j then W = a. 

5- If |tt| > 2, tv does not begin with \ti\, and n does not end with 1 then W = tt. 

Observe that if n and a are permutations then 1 n © 1 avoids a if and only if n avoids a. 
For any set T of permutations we write 

7rG/(3412,T) 

and 

F-( x ) = {-iy i9nM xW. 

7re/(3412,T) 

Egge has shown [TJ Corollary 5.6] that if 7r = cui ffi • • • ffi ct^ is a permutation and ax, . . . , ay. 
are direct sum indecomposable then 

k 

F+(x) = 1 + xF+(x) + x*J2 " C -^ W) ^e-e a ». (10) 

i=i 

where /5 = tt if cti 7^ 1 and (3 = a 2 © • • • © a^ if ax = 1. Our main result in this section is a 
similar recurrence relation for F~(x). Combining this with (|1U|) allows one to compute the 
generating function for the even (or odd) involutions in 7(3412, 71*) for any permutation tt. 

Theorem 4.3 Suppose tt — ax® ■ ■ ■ © aj, is a permutation, where ax, ■ ■ ■ ,a^ are direct sum 
indecomposable. Then 

k 

F-(x) = 1 + xFp{x) -x 2 J2 (^«^) ~ F ^-^M (11) 

Here (3 = tt if ax 7^ 1 and (3 = a 2 © • • • © a^ if ax = 1 ■ 

Proof. The set 7(3412, 7r) can be partitioned into three sets: the set Ax containing only the 
empty permutation, the set A 2 of those involutions which begin with 1, and the set A3 of 
those involutions which do not begin with 1. 

The set Ax contributes 1 to the desired generating function. 

In view of Proposition I2.4f i). the set A 2 contributes xF7(x) to the desired generating 
function, where (3 = tt if ax 7^ 1 and f3 — a 2 © • • • © au if ax — 1. 

To obtain the contribution of A% to the desired generating function, we first observe that 
in view of Proposition I2.4f ii) . all permutations in A 3 have the form ax* a 2 . Since each ctj is 
direct sum indecomposable, if ax*a 2 contains a subsequence of type a* then that subsequence 
is entirely contained in either 19^101 or ff 2 . As a result, the set of involutions which avoid 
3412 and tt and which do not begin with 1 can be partitioned into sets Bx, ■ ■ ■ , Bk, where Bi 
is the set of such involutions in which ax contains ax © • • • © a^x but avoids ax © • • • © flj. 
Since 7 n (3412, «!©•••© a^-i) C 7 n (3412, ax © • • • © aij), and since the sign of ax * o 2 is the 
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negative of the product of the signs of o\ and a 2 , the contribution of the set A 3 to the desired 
generating function is 



-x 2 V ( f~. - (x) - F^—^ (x)) F~ 

i=l 



Add the contributions of Ai, A 2 , and A 3 to obtain (JTTJ). □ 

Theorem 14.31 above is an analogue of [TJ Corollary 5.6], which is a special case of |TJ 
Theorem 5.5]. Using Egge's techniques, one can also prove an analogue of PjJ Theorem 5.5]. 
To state this result, we first set some notation. 

Definition 4.4 Let T = {71*1, . . . , 7T TO } denote a set of permutations and fix direct sum inde- 
composable permutations a* ; 1 < i < m, 1 < j < k i} such that 7Tj = a\ © ■ ■ • © a\. . For all 

ii,...,i m such that < i 3 < k j} let T[* 9h * im = {o£ © ■ ■ ■ © a l ki , . . . , a? m © • ■ ■ © a^ m } . For any 
subset Y C {1, . . . , m}, set 



T Y =\J{ai®-~®°i-i} U W©---©4>- 

jeY j0r,i<j<m 

Theorem 4.5 With reference to Definition ^. J\ 

F^(x) = l+xF- (T) (x)-x 2 E (-^ lYl F TY (x) 

h,...,i m =l \yc{l,2,..,ra} 



^ rpright \X ) 
il,...,im 



Here /?(7Tj) = 7Tj if a\ 7^ 1 ; /3(7Tj) = a^©- • -©o;^. i/ctj = 1 ; and /3(T) is the set of permutations 
obtained by applying (3 to every element of T. 

We omit the proof of Theorem 14.51 for the sake of brevity. 

For the remainder of this section we use to find F~(x) for various n. Combining 
these results with Egge's expressions for various F+(x) allows one to find the generating 
function for the even (or odd) involutions in 7(3412, 7r) for these 7T. We express all of our 
generating functions in terms of Chebyshev polynomials of the second kind, so we begin by 
recalling these polynomials. 

Definition 4.6 For all n we write U n (x) to denote the nth Chebyshev polynomial of the 

sin((n + l)t) 

second kind, which is defined by U n (x) = for n < and U n (cost) = for 

smt 

n > 0. These polynomials satisfy 

U n (x) = 2xU n - l (x)-U n - 2 (x) (n^O). (12) 
We will often use two specializations of U n (x), which are defined for all n by 

1 — x s 



Vn = V n { X ) = U n 

and 

W n = W n (x) = U n 



2x 

1 — x 

2ix 



Observe that by (fT2j) we have 

xV n (x) 

and 



(l - x)K-iO) - xK_ 2 (x) 



zxiy„(x) = (1 - x)W n - 1 (x) - ixW n . 2 (x). (13) 

Turning our attention to F*(x) and F~(x), we now consider permutations of the forms 
\j]ene\j], and tt [j]. 

Proposition 4.7 For any permutation n we have 

^ie7rei( x ) = 



x 



and 



F le7Tel {x) 



x 2 F+(x) 



1 



Moreover, if F+(x) 



for polynomials fo and f\ then for all j > 1. 



1 — x + x 2 F n (x) 
fo and f\ then fo 

+ = fi(x)V^i(x) - xf Q (x)Vj- 2 (x) 

ble^rebl^ > xf^Vjix) - x 2 f Q (x)Vj-i(x) ' 

Similarly, if F~(x) = for polynomials f and fi then for all j > 1, 

F - , * _ fi^Wj-^x) - ixf (x)W j _ 2 (x) 
\j]o*e\j]\ x ) ~ ixf^Wjix) + x 2 f (x)W j . 1 (x)' 



(14) 
(15) 

(16) 



(17) 



Proof. To prove (jTBj) , replace % with 1 % 1 in (jTTj) , use the fact that 1 it 1 is direct 
sum indecomposable and /5 = l07r0lto simplify the result, and solve for F{~ enel (x). 

To prove (fTTj) we argue by induction on j. When j = 1 the result is immediate from 
so we assume j > 2 and the result holds for j — 1. Replace it with [j — 1] % [j — 1] in 
(JT3J), use induction to eliminate F^i^gu Ji) on the right, and use (fT3*j) to simplify the 
result and obtain (JT7j). 

The proofs of ()14j) and (fTo^) are similar to the proofs of ()15j) and (fTTj) respectively. □ 

Arguing as in the proof of Proposition 14. 7\ one can show that if n does not end with 1 
then (frH) - (fT7|) hold when 1 7r 1 (resp. [j] % [j]) is replaced with 1 7r (resp. [j] ix). 
Similarly, one can show that if tt does not begin with \n\ then (fT4"|) - (fl"Tj) hold when 1 n 1 
(resp. [j] 7r [j]) is replaced with 7r 1 (resp. % [j]). 

With Proposition 14.71 and its analogues in hand, we are ready to compute F~ (x) for 
various n. We begin with tt = [k]. 

Theorem 4.8 For all k > 1 we have 



F[2k] ( x 



W k .Ax) 



and 



^[2fc-l]( X ) 



ixWk{x) 

Wk-2{x) - ixW k - 3 (x) 
ix (yV k -i{x) - ixW k ^ 2 (x)) ' 



(18) 
(19) 
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Proof. To prove (fTHj). set % = and = k in (fTtjj) and use the fact that (x) = j. 
The proof of ()19|) is similar to the proof of (|18p. □ 

Remark Lines ()18j) and ()19j) can also be obtained using the methods of Sectional below. 

Using the analogues of Proposition 14. 71 along with the fact that F± 2 (x) = we obtain 
the following result. 
Theorem 4.9 For all k > 1 we have 

W 02311 J r Wei2 W 2X ((1 + x 2 )Wk ( x) _ fa . (1 + x )W k - 1 {x)) ' 1 J 

Remark The fact that F^, Q231 (x) = F^ Q12 (x) is immediate from QjJ Theorem 6.3], which 
implies that I n (3412, [k] 231) = 4(3412, [k] 12) for all n > and all k > 3. 



We now turn our attention to F ffclo213 { x )-> F lk]Q132 (x), and F„j el23 (a; 

-ixW h -i(x) 



[fc]e2i3V x ^ 1 [fc]ei32v x /' miU 1 [fc]ei23 
Theorem 4.10 For all k > we have 



F [k]e2i 3 ( x ) F [k]ei32( x ) ix ( Wk+1 ( x ) _ z X W fc (x)) " (21) 

Proof. First observe that if we apply the reverse complement map to [k] Q 213 and take 
the inverse of the result we obtain [k] 132, and that these operations preserve parity, so 

-^[fc]e213( X ) = -^[A:]el32( X )- 

To show that F^\ Q132 (x) is equal to the quantity on the right, first set n = 132 in (fTTj). use 
the fact that F 2 Ax) = t^— , and solve the resulting equation to find that Fr 32 (x^ - 1 



21 W — l-x-i allKJ - ^ iv ^ lrauum & c 4 uauu " - 1 132V^; _ i_a; +a ;2- 

Therefore (|21|) holds for k = 0. To see that the result holds for k > 0, first set j = k and 
7r = 132 in the analogue of (|T7|) in which [j] n Q [j] is replaced with [j] 7r. Then use (|13|) 
and the fact that F^ 2 (x) = 1 _ :E 1 +3 .a to simplify the result. □ 

Theorem 4.11 For all k > we have 



X 



2\3 



W fc _i(x) - ix(l + 3x 2 + 2x 4 + x 5 )W k - 2 (x) 



W9123V ) ix ^ + x 2) 3V( / fc ( x ) _ ix (l + 3x 2 + 2x 4 + ajS)^^^)) ' 

Proof. This is similar to the second half of the proof of Theorem 14.101 □ 

Recall that 213 and 123 are examples of layered permutations, which are defined as 
follows. 

Definition 4.12 Fix n > 1 and let ■ ■ ■ ,lm denote a sequence such that U > 1 for 

m 

1 < i < m and ^ Zj = n. We write l 2 , ■ ■ ■ , l m ) to denote the permutation in S n given by 
i=i 

[h,l 2 ,---,lm] = [h]®---®[lm]- 

We call a permutation layered whenever it has the form [li , . . . , l m ) for some sequence li,...,l m 
Observe that if m > 2 then [h,...,l m ] = [h, . . . , l m \. In view of ffTTj). IjlBj). and (|T9]l. the 
generating function z , (x) can be expressed in terms of Chebyshev polynomials of the 
second kind for any layered permutation [li, . . . ,l m ]. To do this for m = 2 we will use the 
following well-known identity for Chebyshev polynomials. 



11 



Lemma 4.13 For all k, I > — 1 and all w > we have 

(22) 

It will also be useful to record the recurrence relations for F^Jx). 
Lemma 4.14 For all k > 2 and all I > 1 we have 

1 + xFm (x) 

W = TTl4w (23) 

and 

[Ml J l-x + x^_ 2] (x)+x^(x)- 1 J 

Proof. To prove ()23|). set n = [1,1] in (fTT|) and solve the resulting equation for F^Jx). 
The proof of (}2~4"j) is similar to the proof of (|23p. □ 

We now compute « (x) when and Z are not both odd. 
Theorem 4.15 For all k, I > 1 suc/j Z/iaZ A; and I are not both odd we have 

F m (x) = F [k+l] (x). (25) 

Proof. We consider four cases: k — 1 and Z is even, and Z are both even, k is even and I 
is odd, and k > 1 is odd and Z is even. All four cases are similar, so we only give the details 
for the case in which k > 1 is even and Z is odd. 

In ()24j) replace k with 2 A; and Z with 21 — 1, use (|18|) and (|19p to write the result in terms 
of Chebyshev polynomials, clear denominators, and use (|13|) in the resulting denominator to 
obtain 

[2fc,2i-i] W fx (Wfc _ i xWz _ 2 ) _ Wk _ x ( Wl _ 2 _ ; x w,_ 3 )) ' 

Now apply (}2*2*|) to the numerator and denominator to obtain the right side of ()19j) with k 
replaced by k + I, as desired. □ 

Next we compute i*kn(x) when k and Z are both odd. 
Theorem 4.16 For all k, I > 1 we have 



p- ( \ Wk+j-^x) - 2iW k+ i- 2 (x) - W k +i-z{x) 

ix(W k+l (x) - 2tW k+l - 1 (x) - W k+l ^{x)Y 



Proof. We consider two cases: k — 1 and k > 1. These are similar, so we only give details 
for the case in which > 1. 

In (J24J) replace k with 2 & — 1 and / with 21 — 1, use ()19|) and (|13|) to write the result in 
terms of Chebyshev polynomials, and clear denominators to obtain 

M (W fc -i - iWfc-gXWj - »WU) - (W fc - 2 - zW fc _ 3 )(Wi-i - iWt-g) 
V-i,2i-i]W (l - a;)^ - ixZ 2 - ixZ 3 
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Here 

Z x = - iW k - 2 )(Wi - 

Z 2 = - iW k s)(Wi - iW,_i), 

and 

Z 3 = {Wi-x - iWi-MWk-! - iW k - 2 ). 

Group terms in the numerator and denominator according to the power of i contributed by 
factors of the form iW n to obtain 

. . B 1 -iB 2 + i 2 B 3 

Fr Jx) 



[2fe-l,2l-l]W d - iC 2 + i 2 C7 3 ' 

where 

B 1 = W k . i W l -W k - 2 Wi- 1 , 
B 2 = W k . 2 W x + W fc _iW,_i - W k - 2 Wi- 2 ~ W fc _ 3 Wi_ ls 
S 3 = W fc _ 2 Wj_i - W fc _ 3 Wi-2, 
Ci = (1 - x)W k - X Wi - ixW k - 2 W t - ixWi-iWk-!, 
C 2 = (l-x) (W fe - 2 Wi + W fc _iW,_i) - ix(W fc _ a + W fc _ 3 W,) - ix(f w ^- 2 + W^W^x), 
and 

C 3 = (1 - x)H/ fc _ 2 H/ / _ 1 - ixW^Wlx - ixWi_ 2 W fc _ 2 . 



Apply (j22j) to Si to find Si = Wfc+/-i- Similarly, S 2 = 2iW k+i - 2 and S 3 = W fc+ z_ 3 . 
Now apply (fT3J) and ({22} to G\ to find Ci = ixW k +i- Similarly, C 2 = ix2iW k +i-x and 
C 3 = ixWk+i-2- Combine these results to obtain (J2H|) . as desired. □ 

When m > 3 the generating function F^T i m ](x) does not reduce quite as nicely as it 
does when m = 2. For example, using the same techniques as in the proof of Theorem 14.151 
one can prove that for all ki,k 2 ,k 3 > 1, 

p- / x = W kl+k2+hi (x)W kl+k2+ks -x{x) + Wfe 1+fc2 -i(a;)Wifc 1+ fe3-i(a;)Wii !2 +fc 3 -i(a:) 

(27) 

Nevertheless, (|23j) . (j2T?j) . and (}2*T|) suggest the following conjecture. 



Conjecture 4.17 For all m> 1 and all lx, ■ ■ ■ ,l m > 1, the generating function F„ j , 
symmetric in lx, . . . , l m . 

We have verified this conjecture in the case m = 3 for lj < 24 and in the case m = 4 for 
/j < 20 using a Maple program. 

Egge has conjectured [TJ Conjecture 6.9] that F^ i m ]( x ) is a l so symmetric in lx, ■ ■ ■ , l m - 
This suggests the following conjecture. 

Conjecture 4.18 For all m > 1 and all lx, ■ ■ ■ ,l m > 1, the generating functions for the 
even involutions in 7(3412, [lx, . . . , l m ]) and for the odd involutions in 7(3412, [lx, . . . , l m ]) are 
symmetric in l\, . . . ,l m . 
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5 Involutions Avoiding 3412 and Containing [k] 



In Egge finds the generating function for the involutions in 7(3412) which contain exactly 
r subsequences of type [k]. In this section we prove the following refinements of this result, 
which give the signed analogue of Egge's generating function. Throughout we adopt the 
convention that ("J = 1 and f_"J = for any integer a. 

Theorem 5.1 Fix r > 1, k > 1, and b > such that 

'2k + 2b + 2\ (2k + 2b\ (2k + 2b + 1 
r < rain I I „, , . + 



2k- 1 7 V 2k- 1 / V 2k -1 



Then 

^_iyim(n) x \n\ 

7re/(3412) 



En 



d s + 4+i -/.s -l\ (d s+1 + l s \ WfC X^i -j-,/„+ v (2 , /)+/() 



b 

I ( Cls + 

d s+1 + l s J\ l s )w* 0+> 



b b 

£ 2d j +d -l -1-cfo+E 
p=0 jr 2=0 



Here the sum on the left is over all involutions in 7(3412) which contain exactly r subse- 
quences of type [2k] and the sum on the right is over all sequences do, ■ ■ ■ , db and Iq, . . . , Z& of 
nonnegative integers such that 

-S^ri-r^ri-r 1 ))^^- (28) 

Theorem 5.2 Fix r > I, k > I, and b > such that ( 2fc + 2b ) < r < ( 2k+ ^ +2 ) ■ Then 

^ (-iyi9n(*) x \-K\ = 
7rGi"(3412) 

- 'd a + d s -! +l s -l\ (d a + l s \ Wt° +h ~ l -1-*+ E 2d, -l-do-k+ E 

' ' ' " j=0 



s=0 



Here the sum on the left is over all involutions in 7(3412) which contain exactly r subse- 
quences of type [2k] and the sum on the right is over all sequences d , . . . , d^ and l , . . . ,l b of 
nonnegative integers such that 



r 



■X. ((2k + 2j + 1\ (2k + 2]\\ A, (2k + 2j 



For notational convenience we set d-\ = 1. 

In order to prove these results, we first recall the relationship between J n (3412) and the 
set M. n of Motzkin paths of length n. These are the lattice paths from (0, 0) to (n, 0) which 
contain only up (1,1), down (1, —1), and level (1, 0) steps and which do not pass below the 
line y = 0. We begin by defining a family of statistics on the set Ai of all Motzkin paths. 
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Definition 5.3 Suppose tt is a Motzkin path. For any step s G tt we write ht(s) to denote 
the height of s, which is the x-coordinate of the left-most point of s. For any k we write 




where the first sum on the right is over all up and level steps in tt and the second sum on the 
right is over all down steps in tt. Here we use the convention that (™) = whenever k < 
or k > n. 

Next we recall a r^-preserving bijection between J n (3412) and A4 n . 

Definition 5.4 For any tt G M. n , we write tp(n) to denote the permutation obtained as 
follows. Number the steps in tt from left to right with 1,2, ... ,n. For each up step at height 
k, find the first down step at height k + 1 to its right and switch the labels of the two steps. 
Then <p(tt) is the involution obtained by reading the resulting labels from left to right. 

Proposition 5.5 Proposition 2.13]) For all n > 0, the map if is a bijection between 
J n (3412) and M. n such that Tk((p(Tr)) = t^tt) for all k and all tt G M.. 

The bijection (p allows us to transfer the sign function from J n (3412) to A4 n . 

Proposition 5.6 For any Motzkin path tt we write U(tt) to denote the number of up steps 
in 7i and we write D(tt) to denote the number of down steps in tt. Then for any tt G M, 

fW+DW = signfy^)). (30) 

Proof. Set k = 2 in flU and use Proposition E31 to find that (-1) D W = f or a n 

tt G M.. Now the result follows, since D(tt) = U(tt) for all tt G M. and sign(rr) = (-l^M) 
for all tt G 7(3412). □ 

We now turn our attention to three families of matrices which will use in proving our 
main results. 

Definition 5.7 For all k > we write to denote the k + 1 by k + 1 tridiagonal matrix 
given by 



A k = 



/ X 


ix 








• 


•• 








o\ 


ix 


X 


ix 





• 


•• 














ix 


X 


ix 


• 


•• 























• 


ix 


X 


ix 

















• 


■■ 


ix 


X 


ix 


\o 











• 


■■ 





ix 


x J 



We write to denote the k + 1 by k + 1 tridiagonal matrix obtained by replacing the entry 
in the lower right corner of Ak with 0. We write Ck to denote the k + 1 by k + 1 tridiagonal 
matrix obtained by replacing the entry in the upper left corner of Ak with 0. 
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The matrices A k , B k} and C k are closely related to generating functions for various sets 
of Motzkin paths. To describe this relationship, we let M.{r, s,k) denote the set of lattice 
paths involving only up (1, 1), down (1, —1), and level (1,0) steps which begin at a point at 
height r, < r < k, end at a point at height s, < s < k, and do not cross the lines y = k 
and y = 0. Similarly, we let J\f(r, s, k) denote the set of lattice paths in A4(r, s, k) which do 
not have any level steps at height k, and we let C(r, s, k) denote the set of lattice paths in 
Ai(r, s, k) which do not have any level steps at height 0. Modifying the proof of |3J Theorem 
A2] slightly, we find that 



E 



ir£M(r,s, k) 



E 



n£j\f(r,s,k) 

and 



E 



det(I-A k ) ' {61) 

9 u { n) + D(«) r \n\ _ (-l) r+s det(I-B k ;s,r) 

X ~ det(/ - B k ) ' {62) 

X ~ det(/-C fe ) • {66) 

Here \tv\ is the number of steps in 7r, the quantity U{ji) (resp. D(tt)) is the number of up 
(resp. down) steps in tt, the matrix / is the identity matrix of the appropriate size, and 
det(M; s, r) is the minor of the matrix M in which the sth row and rth column of M have 
been deleted. We will use the fact that the determinants in (JHH), (JH2I), and can be 
expressed in terms of Chebyshev polynomials of the second kind. In particular, arguing by 
induction on k we find that for all k > 0, 

(ix) k+l W k+1 (x) = det(J - A k ) (34) 

and 

{ix) k+l (W k+1 (x) + W k {x)) = det(J - B k ) = det(J - C k ). (35) 

We now prove Theorem 15.11 
Proof of Theorem 1,5. il For any Motzkin path n, let the weight of each up or down step 
in 7r be ix, let the weight of each level step be x, and let the weight of 7r be the product 
of the weights of its steps. Observe that in view of ()30|) and Proposition 15. 5[ the desired 
generating function is the sum of the weights of the Motzkin paths from (0,0) to (n,0) for 
which T 2k (Tr) = r. To compute this generating function, observe that every Motzkin path tt 
with T2 k (n) = r can be constructed by the following procedure in exactly one way. 



1. Choose do, . . . ,db and l , . . . , /{, such that (|2a|) holds. Construct a sequence of down 
and level steps which contains exactly dj down steps at height k + j and lj level steps 
at height k + j for < j < b and which satisfies all of the following. 

(a) The step immediately preceeding a step at height j is either a down step at height 
j + 1 or less or a level step at height j or less. 

(b) All steps after the last down step at height j are at height j — 1 or less. 

(c) The sequence ends with a down step at height k. 
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2. If the first step is at height k +j, insert j + 1 up steps before the first step. Similarly, 
after each step except the last, insert enough up steps to reach the height of the next 
level or down step. 

3. After each down step at height k except the last, insert an (possibly empty) upside- 
down Motzkin path of height at most k — 1 . 

4. Before the first step insert a path from height to height k — 1 which does not exceed 
height k — 1. 

5. After the last step, insert a path from height k — 1 to height which does not exceed 
height k — 1. 

Since the choice at each step is independent of the choices at the other steps, and since every 
sequence of choices results in a path of the type desired, the desired generating function is 
the product of the generating functions for each step. 

To compute the generating function for step 1, suppose we have fixed do,...,db and 

E <*s E (dj+h) 

/o, . . • , h] then each of the resulting partial paths will have generating function i J -° x J -° 
To count these paths, we construct them from the top down. That is, we first arrange the 
lb level steps at height k + b; there is one way to do this. We then place the db down steps 
at height k + b so that one of these steps occurs after all of the diagonal steps. There are 
{ db+ it wa y s t° do this. We then place the lb-i level steps at height k + b — 1 so that none 
of these steps immediately follows a diagonal step at height k + b. There are ( db + lb - 1 ^ ways 
to do this. Proceeding in this fashion, we find that the generating function for step 1 is equal 
to 

£ II (* +d / +1 +l ; " *) ( d - + \ + '■W'^ id,+ "\ pe) 

S _Q V S+l "T" S J \ S J 

where the sum on the left is over all sequences do, . . . ,db and lo, . . . , lb of nonnegative integers 
which satisfy (f2~HJ) . In the path obtained after step 2 there is exactly one up step for every 
down step so the generating function for step 2 is equal to 

b 

E d 3 

(ixy=° . (37) 
Using (}3Tj) and (jMj) . we find that the generating function for step 3 is equal to 

w^Y- 1 (38) 



(39) 



ixWk(x) 

and the generating functions for steps 4 and 5 are both equal to 

1 



ixWJx) 



Taking the product of the quantities in (|3l)j). (jUTj), (|3~5j) and the square of the quantity in 
(HIU), we obtain the desired generating function. □ 
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The proof of Theorem 15.21 is similar to the proof of Theorem I5.1[ using (}3*2*)) , (|33J) , and 
(J35J). 

Theorems 15.11 and 15.21 have several interesting special cases; we give two of them here. 
Corollary 5.8 For all k > 1, 



E (-' 



_1 ^signM^lTr 



2 ' 



^7(3412) (^W) 

where the sum on the left is over all involutions in 7(3412) which contain exactly one subse- 
quence of type [2k] . 

Corollary 5.9 For all k > 1, 

V (_iwM x M = _ 

^ 12) ^(^ fc+1 (x) + ^ fc (x)) 2 

where the sum on the left is over all involutions in 7(3412) which contain exactly one subse- 
quence of type [2k + 1]. 



6 Involutions Which Avoid 3412 and Contain Another 
Pattern Exactly Once 

For any permutation n we write 

Gt(x)= * H > 

o-G/(3412;7r) 

and 

G~(x) = (-l) stgn(a) x lal . 

o-e/(3412;7r) 

In this section we give recurrence relations for G*(x) and G~(x) when n is direct sum 
indecomposable and we use these recurrence relations to give G^(x) and G~(x) in terms of 
Chebyshev polynomials for various n. We begin with the case in which it begins with \n\ 
and ends with 1. 

Proposition 6.1 For any permutation -rr we have 

Gt enel (x) = x 2 Gt(x) (F+^Or)) 2 (40) 

and 

G^ el (x) = -x 2 G-(x) (Fr e , el (x)) 2 . (41) 

Proof. To prove (|40|) . first observe that the set 1(3412; 107T01) can be partitioned into two 
sets: the set A\ of those involutions which begin with 1 and the set A 2 of those involutions 
which do not begin with 1. 

In view of Proposition 12. 4f i). the set A\ contributes xGi Qnel (x) to the desired generating 
function. 
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In view of Proposition l2~4T ii). every permutation in A 2 has the form a\*a 2 . Since 107T01 
is direct sum indecomposable, A 2 can be partitioned into two sets: the set Bi of involutions 
in which 1 tt 1 occurs in 1 <j\ Q 1 and the set B 2 of involutions in which 1 tc 1 
occurs in a 2 . The set B\ contributes x 2 G^(x)F^ e7zel (x) to the desired generating function 
and the set B 2 contributes x 2 Gi enel (x)F+(x). 

Add the contributions of A 1: B\, and B 2 and solve the resulting equation for G± enel (x) 
to obtain 



x 2 G^ (x) F^~ Qnel (x) 
19*01 w ~ i- x - x 2 F +( x ) ■ 



^ie7rei( x ) 



Now (gOJ) follows from IfHj) . 

The proof of (|41j) is similar to the proof of (|4*H|) . □ 

Arguing as in the proof of Proposition 16.11 one can show that if n does not end with 1 
then (j4*Uj) and (}4~T]) hold when 1 n 1 is replaced with 1 tt. Similarly, one can show that 
if tt does not begin with \n\ then (|4()jl and (|41jl hold when 1 7r 1 is replaced with 7r 1. 

To complete our analysis of G^(x) and for the case in which tt is direct sum 

indecomposable, observe that there are four possibilities for the form of tt, corresponding to 
whether or not tt begins with \n\ and whether or not tt ends with 1. Proposition 16.11 and 
its analogues address three of these possibilities. To address the fourth, suppose tt is direct 
sum indecomposable, tt does not begin with |7r|, and tt does not end with 1. Then tt = tt 
and by PJ Proposition 5.7] every involution which avoids 3412 also avoids tt. It follows that 
Gt(x) = G-(x) = 0. 

For the remainder of this section we use Proposition 16.11 and its analogues to compute 
G+(x) and G~(x) for various n. In each case we express G+(x) and G~(x) in terms of 
Chebyshev polynomials of the second kind. We begin with Gi 2 (x) and Gi 2 (x). 
Proposition 6.2 We have 

GUx) = (42) 
1 — x z 

and 

G^) = (43) 

Proof. First observe that if a permutation contains exactly one subsequence of type 12 then 
it has the form n (n — 1) . . .i (i + 1) ... 21 for some i, 1 < i < n — 1. Now observe that 
permutations of this form are involutions if and only if n is even and i = | . Now (|4*2*|) and 
follow. □ 



Using our expressions for Gf 2 (x) and G 12 (x), we obtain expressions for G^ Q231 (x), 

^>]ei2( x )' ^>]e23i( x )' an< ^ G^ el2 (x). 
Proposition 6.3 For all k > 1 we have 

, , (1 — x) 

G[k] Q 23i( x ) = G[ k ] Q12 (x) = p — -j-^ (44) 



and 



x 2 (l + x 2 ^ 



G[ k]e231 (x) G [k]el2 (x) {{l+x2)Wk{x) _ ix{l+x)Wk _ i{x)) 2- { [W) 
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Proof. To see that G^ Q231 (x) = G^] el2 (x), first observe that if 7r = [k — 1] 12 then 
[k] 231 = 1 it 1 and [fc] 12 = 1 tc. Now use flU, P3 Theorem 6.3], and the analogue 
of (|4()jl in which 1 it 1 is replaced with 1 n to find that 



^2 

2 



G+ 07rel (x) = x 2 G+(x){F+^ el (x)) 
= x 2 G+(x) (F+ e M) 
= G lew (x), 

as desired. 

To prove (}4"4"j) we show that Gt, el2 (x) is equal to the quantity on the right for k > 2. 
Arguing by induction on k, first observe that when k = the result is immediate from ()42j) . 
Now suppose > 1 and the result holds for k — 1. Set n = \k — 1] 12 in the analogue of 
([40)1 in which 1 7r 1 is replaced with 1 7r, use jH Theorem 6.2] to replace F+{x) with 
J t " 2 ^ , and use induction to eliminate G+(x) and obtain the quantity on the right side of 
(jUJ, as desired. 

The proof of (|4"H|) is similar to the proof of (|44j) . □ 

Next we compute (^^(x), G^ 32 (x), G^ 13 (x), and G 
Proposition 6.4 M^e nave 



132 



X 



and 



G^) = Gf 32 (x) = l _ x _ %2 (46) 



G 213 (x) = G U2 (x) = - l _ X x \ x _ r (47) 



Proof. First observe that 132 is the reverse complement of 213 and that this operation 
preserves parity, so G 213 (x) = Gt 32 (x) and G 213 (x) = G 



x 



We now show that Gf 32 (x) is equal to the quantity on the right side of ()46|1. To do this, 
first observe that the set 7(3412; 132) can be partitioned into two sets: the set A\ of those 
involutions which have the form 1 a and the set A 2 of those involutions which have the 
form (Ti * a 2 . 

If 7T G Ax, so that 7r = 1 a, then a contains exactly one subsequence of type 21 and 
no subsequences of type 132. Since a contains exactly one subsequence of type 21, it must 
have the form 1 ...(« + 1) i . . .n for some i, 1 < i < n — 1. The only permutation of this 
form which avoids 132 is 213 . . . n. It follows that the set A\ contributes to Gf 32 



x 



If 7r e A 2 , so that 7r = o\ * 02, then cr 2 contains no subsequences of type 21, so a 2 = 
12 ... k. It follows that o~i contains exactly one subsequence of type 132. Therefore the set 
A 2 contributes jz^G^ 32 {x) to Gf 32 (x). 

Add the contributions of A\ and A 2 and solve the resulting equation for Gf 32 (x) to obtain 

flU). 

The proof of (|4"Tj) is similar to the proof of ({HI}) . □ 

Using our expressions for Gj^x), G+ 32 (x), G^^x), and G]~ 32 (x), we obtain expressions 
f° r ( ^[fc]e2i3( :E )' G^ el32 {x) , G^ Q213 (x), and Gj fc ] el32 (x). 
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Proposition 6.5 For all k > we have 

G+Jx) = G+Jx) 



r [fcle213W °>]el32\ x / ,-rr ( \ i T / I \\2 

and 



r [fc]e213V x V ^^10132^/ /,,, / \ • m / \\2- 

[W k+1 (x) -ixW k (x)) 

Proof. This is similar to the second half of the proof of Proposition using (J4"6*j) and ()47|) . 



□ 



Next we compute (^^(x) and G 123 (x). 
Proposition 6.6 We have 



and 

x 3 (l — x 2 ) 
(1 + x 



G^) = -jh-^r- (49) 



Proof. To prove (f4"%)). first observe that the set 7(3412; 123) can be partitioned into two sets: 
the set Ai of those involutions which have the form 1 a and the set A 2 of those involutions 
which have the form o~i * a 2 . 

If it G A\, so that 7r = 1 a, then o contains exactly one subsequence of type 12. It 
follows from (|4"2|) that A\ contributes to Gi23( x )- 

If 7T G A2, so that 7r = o~\ * o~ii then we must have [cr 2 | < 1. If \o~ 2 \ — then o~\ contains 
exactly one subsequence of type 123. If | cr 2 j = 1 then o\ contains exactly one subsequence 
of type 12. It follows from (J4*2~j) that A 2 contributes x 2 G5 t 23 (x) + ^3^2 to Gf 23 (x). 

Add the contributions of Ai and A 2 and solve the resulting equation for Gf 23 (x) to obtain 



The proof of (J4*§|) is similar to the proof of (|48p. □ 
Using our expressions for G^{x) an d GT 23 (x), we obtain expressions for G^ Q123 (x) and 

^[k]el23\ X )' 

Proposition 6.7 For all k > 1 we have 



^[fc]ei23( x ) 



x(l + x 2 )(l - 2x + 2x 3 - :r 



A\2 



(1 - x 2 ) 2 ((1 - x + x 3 )^ fe+ i(x) + x(x - l)T4(x))' 
and 

G _ x 3 (l-x 2 )(l + x 2 ) 4 



[A " :! ' " ((l + x 2 ) 3 iy fc (x)-^x(l + 3x 2 + 2x 4 + x 5 )V^_!(x)) 2 ' 
Proof. This is similar to the second half of the proof of Proposition 16. 31 using (}4"5j) and (|49J1 . 



□ 
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7 Involutions Which Contain 3412 Exactly Once and 
Avoid Another Pattern 



In this section we study involutions which contain exactly one subsequence of type 3412. We 
begin by describing the recursive structure of the set of such involutions. We then use this 
recursive structure to find the generating functions 

o-ei"(7r;3412) 

and 

p-( x ) = (-ir 9n(,j) £ H 

<TGi"(7r;3412) 

for various permutations ft. We start with the notion of a crossing in an involution. 

Definition 7.1 Let ft denote an involution. A crossing in tt is a sequence i < j < k < I 
such that Tr(i) = k and ft (J) = I. Each crossing has a corresponding subsequence in tt, given 
by ft(i)ft(j)ft(k)ft(l) . 

As we show next, involutions which avoid 3412 are exactly those involutions which have 
no crossings. 

Proposition 7.2 Suppose tt is an involution. Then tt avoids 3412 if and only if tt has no 
crossings. 

Proof. (=>■) Suppose tt G 7(3412). Observe that if abed is a crossing then its corresponding 
subsequence has type 3412. But tt avoids 3412, so tt has no crossings. 

(<=) Suppose tt is an involution and it (a) ft(b) tt(c) ft(d) is a subsequence of type 3412. 
Furthermore, suppose it (a) < a. Since tt(c) < it (a) < a and ft(d) < 7r(a) < a, we find that 
7r(c) < c and ir{d) < d. It follows that ft(c)ft(d)cd is a crossing, since tt(c) < 7r(d). Therefore 
it (a) > a, and by a similar argument we find ft(b) > b. Now it follows that abft(a)ft(b) is a 
crossing, since ft (a) < ft(b). □ 

We now consider crossings in involutions which contain exactly one subsequence of type 
3412. 

Corollary 7.3 Suppose ft is an involution with exactly one subsequence of type 3412, given 
by ft (a) ft (b) ft (c) ft (d) . Then ft has exactly one crossing, which is abed. 

Proof. By Proposition 17.21 the involution ft has at least one crossing. Moreover, each 
crossing corresponds to a subsequence of type 3412, so ft has at most one crossing. It follows 
that ft has exactly one crossing. In addition, the subsequence corresponding to this crossing 
has type 3412, so the subsequence must be 7r(a)7r(&)7r(c)7r(d). It follows that the crossing is 
abed. □ 

We now describe the recursive structure of J(; 3412). 
Theorem 7.4 Fix ft G J n (; 3412). Then exactly one of the following holds. 

(i) ft begins with 1. 
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(ii) 



m 



iv 



tt does not begin with 1, 1 is not an element of the subsequence of tt of type 3412, and 
this subsequence lies between 7r(l) and 1. 

tt does not begin with 1, 1 is not an element of the subsequence of tt of type 3412, and 
this subsequence lies entirely to the right of 1. 



1 is an element of the subsequence of tt of type 3412. 

Proof. Observe that it is sufficient to prove both of the following. 

(A) If tt does not begin with 1 and 1 is not an element of the subsequence of type 3412 
then this subsequence lies between -yr(l) and 1 or it lies entirely to the right of 1. 

(B) If 1 is an element of the subsequence of tt of type 3412 then tt does not begin with 1. 

To prove statement (A), suppose by way of contradiction that 7r(a)7r(£>)7r(c)7r((i) is the 
subsequence of type 3412 and 1 lies between n(a) and rc(d). By Corollary 17.31 we have 
7r(a) = c and n(b) = d. It follows that at least one of 7r(l)alc and n(l)bld is a crossing in 
ii. But abed is the unique crossing in tt, so c = Tr(a) = 1, which contradicts the fact that 1 
is not an element of the subsequence of tt of type 3412. 

To prove statement (B), observe that if 1 is an element of the subsequence of type 3412 
in tt then it is the third element of this subsequence, so it cannot be the first element of tt. 
□ 

Definition 7.5 For any permutations txx, tti, and tt 3 we write tt\ <g> tt 2 <S> vt 3 to denote the 
permutation in S , [ 7ri i + | 7r2 [_|j 7r3 | + 4 which is given by 



(7ri®7r 2 ®7r 3 )(i) 



Fl| + |7T 2 | + 3 
7Ti(« - 1) + 1 

kil + K2I + |vr 3 | + 4 



1 

TT 3 (i - 

1 + ki 



Fi 



^1 



2) + 1^1 + 2 
3) 



Fa 



^1 



F2 



if 1 = 1 

if2<%< ki| 
if % = ki| +2 
«/3 < % 
ifi 

if 4 < i 
ifi 



1 



7Ti 



ki| < k2 
+ k2 



ki 



f 3 

7T 2 I < 



F3 



ki 



|7T 2 I + k 3 | + 4 



Theorem 7.6 



(i) For all n > 1, the map 
4-i(;3412) 



TT 



403412) 

1 © TT 



is a bisection between J n _i(;3412) and the set of involutions in 7„(;3412) which begin 
with 1. 

(ii) For all n > 1 and all j such that 2 < j < n, the map 

7 3 -_ 2 (;3412) x 4^(3412) — > 7 n (;3412) 

(7Ti,7T 2 ) 7rx*7T 2 

a bisection between 7j_ 2 (;3412) x J n _ :) '(3412) an<i t/ie set of involutions in 7 n (;3412) 
which do not begin with 1, in which 1 is not an element of the subsequence of type 3412, 
and in which this subsequence lies between tt(1) and 1. 
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(iii) For all n > 1 and all j such that 2 < j < n, the map 

7 i _ 2 (3412)x /„_,-(; 3412) — > 7 n (;3412) 

(7Ti,7r 2 ) l-> 7Ti * 7T 2 

zs a bijection between 7 ? _ 2 (3412) x 7 n _j(3412) and t/ie sei of involutions in 7 n (;3412) 
which do not begin with 1, in which 1 is not an element of the subsequence of type 34.12, 
and in which this subsequence lies entirely to the right of 1. 

(iv) For each k > 0, set [k] = k k — 1 ... 21. For all n > 1 and all j, k, m > such that 
0<j + k + m<n — 4, the map 

7^(3412) x 4(3412) x 7 m (3412) — > 7 n (; 3412) 

(■7Ti, 7T 2 , 7T 3 ) h-> (7Ti (g) [n - j - fc - m - 4] <g> 7T 2 ) © 7T 3 

zs a bijection between 7,(3412) x 7^(3412) x 7 m (3412) and the set of involutions in 
7 n (;3412) in which 1 is an element of the subsequence of type 3412. 

Proof, (i) This is immediate from the fact that if tt begins with 1 then 1 cannot be an 
element of a subsequence of type 3412 in tt. 

(ii) Observe that if the subsequence of type 3412 lies between and 1 then the elements 
of 7r to the left of 1 are 2, 3, ... , 7r(l). Now (ii) follows. 

(iii) This is similar to the proof of (ii). 

(iv) It is routine to verify that every permutation of the form given contains exactly one 
subsequence of type 3412, and that 1 is an element of this subsequence. 

To show that the given map is one-to-one and onto, suppose tc G 7 n (;3412) and that 
1 is an element of the subsequence of type 3412 in tt. Let 7r(a)7r(6)7r(c)7r(d) denote this 
subsequence. Then there exist unique sequences cr , c^, a 2 , cr 3 , cr 4 which avoid 3412 such that 
7r = aoaa\ba2ca^dai. Since 1 is an element of 7r(a)7r(6)7r(c)7r(d) and this sequence has type 
3412, we have 7r(c) = 1. By Corollary 17.31 we have 7r(l) = a, so a is empty. 

Now set j = | eri|, k — [cr 3 1 , and m = | cr 4 1 , so that [ cr 2 j = n—j — k — m — A. Since abed is the 
only crossing in tt, the entries of <J\ are 2, . . . , j + 1, the entries of cr 2 are 7 + 3, . . . , n — k—m — 2, 
the entries of 03 are n — k — m, . . . ,n — m — 1, and the entries of 04 are n — m + 1, . . . , n. 
Moreover, if a 2 has a subsequence 7r(e)7r(/) of type 12 then 7r(a)7r(6)7r(e)7r(/) will have type 
3412. It follows that cr 2 has type [n — j — k — m — 4]. Combining these observations, we 
find there exist unique permutations 71*1,7^, 7T3 such that |7i"i| = j, \tt 2 \ = k, \tt 3 \ = m, and 
7r = {1x1 ® [n — j — k — m — 4] ® 7r 2 ) © 7r 3 . It follows that the given map is a bijection, as 
desired. □ 

Using Theorem 17.61 we find the generating function for those involutions which contain 
exactly one subsequence of type 3412. 
Proposition 7.7 We have 

P+(x)= l X ~\ + l -2x-2x 2 
01 ; 2x\l-x) 2xVl - 2x - 3x 2 1 ; 

and 

p - r x _ (x + l)(2x 2 - 2x + 1) (x 2 - l)(4x 2 - 2x + l) 
01 J 2x 2 (l + x 2 ) 2x2(1 + x 2 ) v / 5a; 2 -2x+r 1 J 
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Proof. To prove ()5U|) , first use Theorems 17.41 and 17.61 to find that 

Pf(x) = xP+ix) + 2x 2 P+(x)F+(x) + X4( f^ (x))3 - 
Now use the fact that 

. 1 — x — — 2x — 3x 2 
F* (*) = ^ 

to eliminate Ft{x) and solve the resulting equation for Pn~(x) to obtain ()50|) . 
The proof of (|51|) is similar to the proof of f|50j) . using 



P -(x) = ^"(a;) - 2x 2 P -(x)F -(a;) H 

and 



x 4 (l + x)(F -(x)) 3 
n i' r "'n +" Y+x~ 2 



_ . . x — 1 + V5x 2 — 2x + 1 

W = ^ ■ 

□ 

For the rest of this section we use Theorem 17.61 to find P^(x) and P^(x) for various ir. 
We begin with the case in which it = [k]. 
Proposition 7.8 For all k > 1 we /tai>e 

_ e-: 2 (i-^ +2 )^ ) 

i 2fc i lxJ (i-x)y fc 2 (x) 

and 



^1^)-— ( i_ x)l ,2 (x) ' V " ( 52 ) 



(v k+1 (x) + v k ( x )) 

Proof. To begin, use Theorems 17.41 and 17.61 to find that 



xP+(x) + x 2 P [k _ 2] (x)F+ k] (x) + x 2 F+_ 2] (x)P+(x 



+ x + ■ ■ ■ + x^) [F+_ 2] (x) j F+(x). 

Now solve this equation for P^Jx), use [TJ Theorem 6.1] and argue by induction on k to 
obtain (jH2J) and (jHS))- □ 

Proposition I7.8l enables us to find |4([&]; 3412) | for various k. We obtain simple formulas 
for this quantity when k = 3 or k = 4. 
Corollary 7.9 For all n > A, 

^ 7?, 1 

|/ n (321; 3412)| = — z-i^ + -— F n _ 3 
5 5 

and 

|4(4321; 3412)| = 2 n - 5 (3n - 10). 

Here F n is the nth Fibonacci number, which is defined by F = 0, F% = 1, and F n = 
F n _ 1 + F n _ 2 forn>2. 
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Proof. Set k = 1 in fl33J) and set k = 2 in fl^U). □ 

Arguing as in the proof of Proposition I7.8[ we also obtain PjZ, (x) 
Proposition 7.10 For all k > 1 we have 



[2k] 



E£X-(s)m?0»0 



and 



(^ fe+1 (x)-z^(a;)) 2 



^[2fc+l] ( X ) 



(54) 



(55) 



#ere u fc (x) = ££=(,(- 1)W a?* f or allk>0. 

Propositions 17.81 and 17.101 include P^i(x) and P^ 2 i{ x ) as special cases. We now use 
Theorem 17. 61 to compute P^{x) and P~(x) for all remaining permutations of length 3. 
Proposition 7.11 We have 



71 


P+(x) 




123 


x 4 (l + X + x 2 ) 


x 4 (l + 3x- 3x 2 + 6x 3 + 3x 4 + 3x 5 - x 6 ) 


(l + x)(l-x) 3 


(l + x 2 )(l + x) 3 (l-x) 3 


132,213 


4 

X 


x 4 (l + x) 


(1 — x)(l — x — X 2 ) 


(1 -x + x 2 )(l + x 2 ) 


231,312 









Proof. The last line of the table is immediate, since a permutation which contains a subse- 
quence of type 3412 must also contain subsequences of type 231 and 312. 

The fact that Pit Q (x) = P^ix) is immediate, since 213 is the reverse complement of 132. 



To obtain P, 



132 



132 V 

'x) 



^132 \ X 



use Theorem 17.61 to find that 

^ PUx) + x 4 



1 — x 1 — X 



Solve this equation for P^^x) to obtain the desired result. 
The rest of the table can be obtained in a similar fashion. □ 



Building on Proposition 17.111 we now compute P ] 
length 2 and 3. We begin with n = 12. 
Proposition 7.12 For all k > 1 we have 



[k]en {x) and P [k]QiT (x) for various n of 



and 



Here Vk(x) = £ 



-^ > [fc]ei2( x ) 



ELI Vj-i(x) (Wj^ix) - ixf (x)Wj^(x)y 



;i -x)i4 2 +1 i 



;r 



-^ > [fc]ei2( x ) 



, f|l 1)®^ /or all k > and / (x) = 

26 



(56) 



(57) 



c+2^2 ' 



Proof. To prove ()56|) we argue by induction on k. The case k = 1 is immediate from 
Proposition 17. Ill so suppose k > 2 and the result holds for k — 1. Use Theorems 17.41 and 17.61 
to find that 

^ 2^ ^ / _|_ \ 2 



Solve this equation for P^ el2 ( x ) and use induction and the fact [TJ Theorem 6.2] that 

F i]en( x ) = Jgf) to obtain ©. 

The proof of (JoTj) is similar to the proof of (jo^|) . □ 

We now turn our attention to P^ e i 32 (x) an d P\k]e2is( x ) • 
Proposition 7.13 For all k > 1 we have 

p+ ( ( s (1 - x 3 ) (V 2 (x) + KQe)) + ZU gjfr) + ^-i(^)) 2 ^ 

[ * 132(X] ~ [fcle213( " j ~ (1 - x) ( W*) + F fc+1 (x)) 2 • (58j 

Proof. It is immediate that P^ Q132 ( X ) = P\k]Q2is( x ) > srnce [&]0132 is the reverse complement 
of [fc] 213. To prove that -P[fc] el32 ( 2; ) is equal to the expression on the right side of (p)H|) we 
argue by induction on k. 

To prove the result when k — 1, use Theorems 17.41 and 17.61 to find that 

P^32i x ) = 

x Pil32( x ) + x A~32 ( x ) -^4132 ( x ) + x -^132 ( x )-^4132 ( x ) + 

x * ■ (FiU x ) - 1) • 1 ■ 1 ■ F+ 32 (x) + x 4 ■ 1 • -i- ■ 1 ■ F+ 32 (x). 

Solve this equation for P£i 3 2( x ) and use Proposition 17. 1 II and the fact [TJ Theorem 6.4] that 

F i32i x ) = !-Lx* and F 4i32( x ) = X -2x X -x*\x* to find that © holds when k = \. 

Now suppose k > 2 and (fo^j) holds for A; — 1. Use Theorems 17.41 and 17.61 to find that 

F [k]el32( X ) = 

xF \k]ei32( x ) + x ^ , [fc-i]ei32( x )-^'[fc]ei32( a ') + x F \k-i}ei32( x ) F \k]ei32( x ) + 

X F \k-l]el32( X ) ' I _ x ' F [k-l]el32( X ) ' F [k]Q132( X ) 

Solve this equation for P^ el32 (x) and use induction and [TJ Theorem 6.4] to obtain (j5SJ). □ 
In order to find P^ e23 i( x ), we first consider a more general situation. 
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Proposition 7.14 For any permutation tt we have 

Pi + e„ei(x) = (i^eiO*)) 2 fax) + ^ (#(*)) ' 

JW(*) = -* 2 (*W*)) 2 fax) - K"W) 2 ) 

Proof. To prove (|59jl . use Theorems 17.41 and 17. 61 to find that 



and 



(59) 
(60) 



4 

.X 



(1 - x - x 2 F+(x)) P^^x) = x 2 F+(x)P+(x) + — (F+(x)Y F^ Ql (x). 

Now (JSni) follows from (fTIjl . 

The proof of (|60j) is similar to the proof of (|5T?j) . □ 

Arguing as in the proof of Proposition 17.141 one can show that if tt does not end with 1 
then (|59|) and (f6T)|) hold when 1 tt 1 is replaced with 1 tt. Similarly, one can also show 
that if tt does not begin with \tt\ then (|59jl and (|6()j) hold when 1 tt 1 is replaced with 

7T0 1. 

Proposition 7.15 For all k > 2 we /iai>e 



Pt^AX 



W e23lV;__ (l-a;)y fc 2 +1 (x) 

Proof. Set 7r = [A; — 1] 12 in (J5T?|) and use ()56|) and Theorem 6.2] to simplify the result. 
□ 

In view of Conjecture 14. 171 and [TJ Conjecture 6.9], one might conjecture that P^ i m \{ x ) 
and P^ i m ](x) are symmetric in li,...,l m . However, it is not difficult to verify (using 
Maple) that this fails for [Z Xj l 2 , h] = [2, 1, 1] and [h, l 2 , h] = [1, 2, 1]. 

8 Involutions Which Contain 3412 and Another Pat- 
tern Exactly Once 

For any permutation tt and any n > 0, let J n (; 3412, it) (resp. J(;3412,7r)) denote the set 
of involutions of length n (resp. of any length) which contain exactly one subsequence of 
type 3412 and exactly one subsequence of type tt. In this section we begin a study of the 
generating functions 

Q+(x) = xWl 

crG/(;3412,7r) 

and 

Q-{x) = (-l)" 9n(<T) x l<T| . 

cre/(;3412,7r) 

It seems possible to obtain results similar to the results of previous sections, but the expres- 
sions involved become cumbersome rather quickly. For this reason we restrict our attention 
to Qf k] (x). 
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Theorem 8.1 For all k > 2 we have 

n+ 2x * (1- x 2 ^ 1 )) ^_ 2 (x)\/,(x) + gg (1 ~ ^ 2(J+1) ) V?(x) f 

Q[2k][X) (1 " *)V k 2 (x) ^ V5_x(x)V5(x) 1 j 

and 



Q[2fc+i]( x ) /-i__.\v2 



(1 - x)Y k \ l {x) F m (x)F i+2 (x) 



i=2 

where Y}.(x) = Vk{x) + V^._i(x) /or a// A;. 
Proof. In view of Theorems 17.41 and 17.61 we have 

X Q~[k]( X ) + 2 ' 2 Q[fc-2]( a ')-^ , [fc]( a; ) + ^ ^-2}( X )^\k]( X ) + 2 ' 2 -^ 1 [fc-2]( a; )Q[A:]( a:; ) 

+x 2 G+„ 2] (x)P+(x) + x 4 ^ j ^G+_ 2] ( : r) J P [ t 2] (x)F [ +(x) + G+(z) (f+_ 2] (x 

Now the result follows by induction on A;, using the fact that Q~^(x) = along with [TJ 
Theorems 4.2, 4.3, 6.1], and □ 

Theorem 18. II enables us to find | J n (; 3412, [fc])| for various fc. We obtain a simple formula 
for this quantity when k — 4. 
Corollary 8.2 For all n > 6 we have 

|J n (; 3412, 4321) | = (3n 2 - 23n + 38)2"" 8 . 
Proof. Set A; = 2 in (jnij). □ 



9 Directions for Future Research 

1. For any m > 1 and any permutation it G 7(3412), let 

oo 
k=l 

Recall that u)i(tt) is both the number of left-to-right maxima in it and the number of 
right-to- left minima in n, and that o; 2 (7r) is the number of fixed points in n. Find a 
combinatorial interpretation of Lu m {^) for m > 3. 

2. In view of ([18)1 . (|20j). and Theorems 6.1 and 6.2], the number of even (resp. odd) 
permutations in 7 n (3412, [2 A;]) is equal to the number of even (resp. odd) permutations 
in 4(3412, [jfe-l]Q231) for all n > 0. Similarly, in view of (HHJ), (J2U), and [H Theorems 
6.1 and 6.4], the number of even (resp. odd) permutations in 4(3412, [2A; + 1]) is equal 
to the number of even (resp. odd) permutations in 4(3412, [k— 1] 132) for all n > 0. 
Give combinatorial proofs of these facts. 
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3. Prove Conjecture 14.171 which says that l Jx) is symmetric in li, . . . , l m . Alter- 
natively, prove Conjecture I4.18| which says that the generating functions for the even 
involutions in 7(3412, [li, . . . , l m ]) and the odd involutions in 7(3412, [h, ■ ■ ■ , l m ]) are 
symmetric in l\, . . . , l m . 
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